<a href="#YNULLDEC">Why Null Decapsulators Work Differently</a>
<a href="#OVERLAPD">Overlapping Decapsulators<a>
<a href="#PARSEMPF">Parsing Empty Fields</a>
<a name="INTRODUC"><h2>Introduction</h2>
A "decapsulator" is a command parameter that defines a search for where a string of characters either begins or ends.
If that definition was not particularly helpful, it is because decapsulators cannot be fully described by a single sentence. But we encourage you to read through this appendix, for the following reason:
<hl>
Decapsulators let a <b>single</b> Parse-O-Matic Scripting command accomplish
what might take <b>dozens</b> of commands in a standard programming language.
</hl>
The underlying concept is this: when analyzing data, the part you are interested in (the "field") is typically surrounded ("encapsulated") by some kind of distinctive text. A decapsulator looks for the distinctive text on either side of the data you want and thus helps you extract the field.
Sometimes the "distinctive text" appears more than once in the data you are examining. Decapsulators can handle this situation.
Sometimes one edge of the field is the beginning or end of the data you are examining, so there is no "distinctive text" to look for. Decapsulators can handle this situation, too.
Each of these techniques is explained below in more detail.
<a name="SIMPLEEX"><h3>A Simple Example</h3>
Here is an example of how decapsulators work. Consider the following commands.
<hl>
SourceVar = 'AAABBBCCC'
ResultVar = Parse SourceVar '3*A' '1*C'
</hl>
The second command means "Set ResultVar to everything between the third occurrence of 'A' and the first occurrence of 'C'.' In other words, ResultVar will end up containing 'BBB'.
<a name="YDECAPNE"><h2>Why Decapsulators are Necessary</h2>
When analyzing data, the fields you are interested in are sometimes arranged in tidy columns ù but not always. Quite frequently, a field will start after some kind of delimiter, as in the following example.
<hl>
SourceVar = 'Mouse,Gazelle,Mouse,Elephant'
</hl>
Here the fields are separated by commas ù a commonly-used format for data known as CSV (Comma Separated Values).
Extracting, say, the second item from free-form data is rather awkward if you are using a standard programming language. Fortunately, Parse-O-Matic Scripting has been developed with precisely this kind of situation in mind.
Using decapsulators, the Parse command lets you extract the "Nth" item. For example, to extract the third item in the free-form example above, you could use this command:
<hl>
ResultVar = Parse SourceVar '2*,' '3*,'
</hl>
This means "Set the variable ResultVar by looking in SourceVar and taking everything between the second comma and the third comma". ResultVar would thus be set to 'Mouse'.
<a name="OCCURENC"><h2>Introduction to Occurrence Numbers</h2>
Let's have another look at that last command.
<hl>
ResultVar = Parse SourceVar '2*,' '3*,'
</hl>
The first decapsulator (i.e. the <hl>'2*,'</hl> part) is the "From" specification. The second decapsulator (i.e. the <hl>'3*,'</hl> part) is the "To" specification. It is interpreted as follows:
<hl>
3 means "the third occurrence"
* marks the end of the occurrence number
, is the text you are looking for
</hl>
Decapsulators can be used to find more than a single character. Let's say that (for some odd reason) a variable named xyz has been set such that each field is separated with a pair of X's, as in the following example.
The occurrence number must always be followed by an asterisk (the * character) so you can search for a number. Consider the following example (the meaning of which would be unclear without the asterisk):
<hl>
MyVar = Parse 'xxx2yyy2zzz2' '1*2' '2*2'
</hl>
This sets MyVar to the text occurring between the first '2' and the second '2'. In other words, MyVar is set to 'yyy'.
<a name="FTFALOCC"><h3>Finding the First and Last Occurrence</h3>
A decapsulator can refer to "the <b>last</b> occurrence":
<hl>
xyz = Parse 'AaaBAbbBAccB' '>*A' '>*B'
</hl>
In both decapsulators, the > symbol means "the last occurrence". Thus, the command means, "Set the xyz variable to everything between the last A and the last B". Thus, the xyz variable is set to "cc".
You can also use the < character to mean "the <b>first</b> occurrence", though this is somewhat redundant, since the following commands are equivalent:
<hl>
abc = Parse 'AaaBAbbBAccB' '<*A' '<*B'
abc = Parse 'AaaBAbbBAccB' '1*A' '1*B'
abc = Parse 'AaaBAbbBAccB' 'A' 'B'
</hl>
All three commands would set the abc variable to 'aa'.
<a name="FNEXTOCC"><h3>Finding the Next Occurrence</h3>
When using occurrence numbers for certain kinds of data, you will often find that the "To" occurrence number is 1 (one) more than the "From" occurrence number. Consider this example:
<hl>
xyz = 'AB,CD,EF,GH'
Field1 = Parse xyz '' '1*,'
Field2 = Parse xyz '1*,' '2*,'
Field3 = Parse xyz '2*,' '3*,'
</hl>
For Field3 you are extracting everything between the second and third comma. It can become tiresome to write code like this ù always adding one to the "From" occurrence number. Fortunately, you can use the "next occurrence" symbol '@*' in the "To" decapsulator:
<hl>
xyz = 'AB,CD,EF,GH'
abc = Parse xyz '2*,' '@*,'
</hl>
This will set the "From" position to the second comma, and the "To" position to the comma after that (i.e. the third one). The '@*' symbol means "Look for the To text starting immediately after the From text".
Note: The "next occurrence" symbol (@*) can only be used in the "To" decapsulator.
Note: Positional decapsulators imply that operations proceed from or to the exact character position indicated, regardless of the <a href="#CONTROLS">control settings</a>.
You can specify a number to indicate the "From" or "To" position. In this mode, the Parse command behaves exactly like the Cols command. Thus, the following two commands accomplish the same thing:
<hl>
xyz = Parse MyVar '10' '20'
xyz = Cols MyVar '10' '20'
</hl>
As such, this is not particularly helpful. However, you can combine positional decapsulators with other types of decapsulators, as in this example:
<hl>
MyVar = 'ABCD/abcd/'
abc = Parse MyVar '3' '1*/'
</hl>
This will set the variable abc to 'CD'.
You can also count backwards from the right edge of the data. Consider this example:
<hl>
MyVar = 'ABCDEFG'
xyz = Parse MyVar '-3' '-2'
</hl>
This will set the variable xyz to 'EF'. (The last character in a variable is represented by position '-1'.)
You need to be careful when you use positional decapsulators. If, for example, you use a negative positional decapsulator, and you end up referring to a character before the beginning of the string, it isn't clear what you "mean" by that.
For the reason just noted, and others that will become evident as you write scripts: if there is a chance that a positional decapsulator will refer to a character position of zero or less, or if it might refer to a position beyond the end of the data, you should check the length of the data before trying the command.
The occurrence number is not always needed. Either the "From" or "To" decapsulator can be represented as a plain (non-numeric) string, as in the following example.
<hl>
OldVar = 'zzzABChelloXYZzzz'
NewVar = Parse OldVar 'ABC' 'XYZ'
</hl>
This would set the variable named NewVar to 'hello' since it means:
<hl>
- Copy from the character following the first 'ABC'
- Copy up to the character preceding the first 'XYZ'
</hl>
This is, of course, equivalent to the following command, which uses occurrence numbers:
<hl>
NewVar = Parse OldVar '1*ABC' '1*XYZ'
</hl>
In general, it is best to explicitly give occurrence numbers, unless you know that the format of the data is not going to change.
<a name="UNSUCCES"><h2>Unsuccessful Searches</h2>
When a command that uses decapsulators does not find the search text, it does as little as possible. For example, if a Parse command does not find the encapsulating text, it sets the variable to a null (''). Here are two examples:
<hl>
abc = Parse 'ABCDEFGHIJ' '1*K' '1*J' <ùù There is no 'K'
abc = Parse 'ABCDEFGHIJ' '1*A' '1*X' <ùù There is no 'X'
</hl>
To illustrate this principle further: if the Overlay command does not find the search text, it does nothing at all, as in the following example.
<hl>
abc = 'ABCDEFGHIJ' <ùù Set a variable
Overlay abc 'K' 'LMNOP' <ùù There is no 'K', so nothing is done
</hl>
If the "From" value is less than the "To" value, the Parse-O-Matic engine will display an error message, then terminate further processing. For example:
This kind of failure typically happens if the data contains an odd arrangement of text that you had not foreseen. In such case, it would not be reasonable for processing to continue; you need to be warned about departures from what your script implies you expected.
<a name="CONTROLS"><h2>The Control Setting</h2>
Commands that use decapsulators typically have a "control setting" that allows you to adjust the way the command is performed. These settings are described in more detail in the "Parse-O-Matic Scripts" user manual, but a few examples can be given here.
The Parse command's control setting tells Parse whether to include or exclude the surrounding (i.e. searched-for) text. By default, the surrounding text is excluded (unless the decapsulator is <a href="#POSITION">positional</a>). However, if you want to include it, you can add 'Include' at the end of the Parse command, as in this example:
<hl>
xyz = Parse 'aXcaYcaZc' '2*a' '2*c' 'Include'
</hl>
This tells the command to give you everything between the second 'a' and the second 'c' ù <b>including</b> the 'a' and 'c'. In other words, this sets the variable xyz to 'aYc'.
You can also set the Control specification to 'Exclude', though since this is the default setting for Parse, it isn't necessary. Here is an example:
<hl>
xyz = Parse 'a1ca2ca3c' '2*a' '2*c' 'Exclude'
</hl>
This sets the variable xyz to '2'.
You can specify several control settings at once, separated by spaces. By default, the Parse command's control setting is 'Exclude MatchCase' but you could set this to (for example) 'Include IgnoreCase'.
<a name="NULLDECA"><h2>The Null Decapsulator</h2>
Here is helpful variation of the "From" decapsulator:
<hl>
'' means "Start from the first character in the value being analyzed"
</hl>
A similar variation can be used with the "To" decapsulator:
<hl>
'' means "End with the last character in the value being analyzed"
</hl>
If you use the null ('') decapsulator for "From" or "To", the "found" value (the first character for "From", or the last character for "To") will always be included (see <a href="#OVERLAPD">Overlapping Decapsulators</a> for an exception to this rule). Here is an example:
<hl>
xyz = Parse 'ABCABCABC' '' '2*C"
</hl>
This sets the variable xyz to "ABCAB". The "From" value (i.e. the first character) is <b>not</b> excluded. However, when Parse finds the "To" value (i.e. the second occurrence of the letter C) it <b>is</b> excluded. If you want to include the second 'C', you should write the command this way:
<hl>
xyz = Parse 'ABCABCABC' '' '2*C' 'Include'
</hl>
Incidentally, the following two commands accomplish the same thing:
<hl>
xyz = Parse 'ABCD' '' ''
xyz = 'ABCD'
</hl>
They are equivalent because the Parse command means "Set the variable xyz with everything between (and including) the first character and the last character".
<a name="YNULLDEC"><h3>Why Null Decapsulators Work Differently</h3>
It may not be immediately obvious why decapsulator-enabled commands treat the null ('') decapsulator differently. The examples given here are very simple, and not representative of real-world applications.
In day-to-day usage, though, you will frequently find it helpful to be able to specify a command that says, "Give me everything from the beginning of the line to just before such-and-such" or "Give me everything from such-and-such a point until the end of the line."
For example, here is a command that means "Give me everything from just after the dollar sign, to the end of the line":
<hl>
xyz = Parse 'Please give me $199.00' '1*$' ''
</hl>
This sets xyz to "199.00". If you want to include the dollar sign, write the command this way:
<hl>
xyz = Parse 'Please give me $199.00' '1*$' '' 'Include'
</hl>
In this example, the 'Include' control setting affects the way the "From" decapsulator works, since it is using an <a href="#OCCURENC">occurrence number</a>.
Earlier, it was mentioned that the text found by the null decapsulator is "always included" and is not affected by the 'Exclude' control setting. There is an exception to this: if the null decapsulator's "found text" is contained in the text found by the other decapsulator, it <b>can</b> be affected. For example:
<hl>
xyz = Parse 'ABCDEFABCDEF' '' '1*AB' 'Exclude'
</hl>
This command means "Give me everything between the first character and the first occurrence of AB". Since the two items overlap (i.e. the first 'AB' includes the first character), the first character does indeed get excluded. As a result, the xyz variable is set to an empty string ('').
Here is another example:
<hl>
xyz = Parse 'ABCDEFABCDEF' '>*F' '' 'Exclude'
</hl>
This command means "give me everything between the last occurrence of F and the last character". Both decapsulators refer to the same character (i.e. the final 'F'), so it is excluded. As a result, the xyz variable is set to an empty string ('').
Note: In some circumstances, the FindPosn command is <b>not</b> affected by this exception. It will do its best to make sense of your request if the decapsulators overlap, and one of them is a null decapsulator. The "Parse-O-Matic Scripts" manual explains this in more detail.
<a name="PARSEMPF"><h2>Parsing Empty Fields</h2>
Consider the following command, which is operating on CSV (Comma Separated Value) data.
<hl>
xyz = Parse ',,,JOHN,SMITH' '2*,' '3*,'
</hl>
There is nothing between the second and third comma, so the xyz variable is set to '' (an empty string).
Now consider this command:
<hl>
xyz = Parse ',,,JOHN,SMITH' '' ','
</hl>
You are asking for everything from the first character to the first comma (which also happens to be the first character). Obviously, there is nothing "between" the two characters, so the xyz variable would be set to '' (an empty string). This may be what you wanted, but whenever you are dealing with a field at the beginning or end of data, and there is a chance the field might be empty, it is a good idea to test your script to make sure that it does what you expect.
